Introduction

In the first section, we examined how a simple biological neuron works and introduced the perceptron model of an artificial neuron, which we have used in its various implementations to build various neural networks. One may recall from those lessons that a fundamental difference between the biological and the artificial neuron is in its output. Regardless of the activation function implemented so far in these notes, the output of an artificial neuron is constant, whereas the output of the biological neuron is a series of spikes that are a more akin to frequency modulated output:

There are several reasons why researchers might want to model the behaviour of biological neurons more closely.

• Firstly, biologists and other life scientists believe that by accurately simulating the function of a neuron they may be able to gain a greater understanding of the human nervous system. This field of science is known as ‘Computational Neuroscience’ and often involves the creation of very large and complex models of the human brain. We noted earlier that the biological neuron introduced earlier was greatly simplified, and it was this simplified model that inspired the basic perceptron artificial neuron. Computational neuroscientists often seek to emulate a more complete and complex model of biological neurons requiring much greater computer processing capability.
• In the field of psychology, some researchers believe the spiking (or pulsing) nature of the neuron is important to the phenomenon of consciousness. Some postulate that it is how these pulses move around and interact within the nervous system that produce the human conscious experience or other forms of higher-level thought. This work has important implications for the moral consideration of non-human agents that have neural networks, e.g, animals or artificial intelligence.

Whilst these fields of study are interesting, their work is outside the scope of these lectures. In general, spiking neural networks have found limited application within the engineering field, but their functioning is still worth understanding.

There are some engineering applications in which the use of spiking neurons can add benefit. For example, in the basic obstacle avoidance robot (see Section 4), the network was trained using a back propagation algorithm, however, the motor actuators driving each wheel would still require timers to drive each wheel for the appropriate period of time. A spiking neuron would mitigate this need as the strength of its output (akin to a pulse wave modulated output) could be used in the drive circuit. In general, spiking neurons may be of benefit to engineering applications that require some form of timing control.

Spiking neurons for timing control applications

As noted above, we are not here going to attempt to accurately emulate the biological neuron but focus instead on using a spiking neuron for timing control applications. As such we are still going to be working with a greatly simplified artificial neuron model. In this model, we wish the output of the neuron to produce a short high (a spike) followed by some period of time at a low value or zero (t_off). Where t_off is large, the neuron is weakly activated; where t_off is small, the neuron is strongly activated:

In this conceptualisation, the short spike is of a fixed length, the ‘strength’ of the neuron activation being controlled by the length of t_off. These neurons are usually configured so that once they trigger and start producing an output, they don’t respond to further inputs until the cycle is finished.

In many implementations, HIGH and LOW are set as follows: HIGH = +1 and LOW = -1.

We can vary this model further by adding control of the length of time that the spike is set at HIIGH (t_on). Alternatively, we can control multiple iterations of HIGH and LOW within the same fixed period of the cycle (T):

In the following section, we shall consider how one might build a network that uses a PWM neuron for robotic drive control.

PWM neuron for robotic drive control

In Section 4 we considered how a network using a back propagation algorithm could be used in autonomous object avoidance in robotic vehicles. In that lesson, we talked about how the algorithm could be used to train the network but we did not discuss how the drive control would actually be implemented. Say for example we have the following scenario:

Figure 3: Obstacle avoidance

Suppose our robotic vehicle is programmed to drive straight forward (1). On detection of an object (on the left bump sensor) we want our robot to reverse and turn right at the same time as marked in red (2). After that action has been taken, our robotic vehicle should continue to drive forward once again (3). We could implement this using a simple artificial neural network:

Figure 4: A drive controller for a simple robot vehicle

Now let’s suppose that each neuron works as follows:

• An inactivated neuron output (-1) drives the wheels forward.
• An activated neuron output (0 to +1) drives the wheels in reverse.

The network, therefore, would operate as follows:

• With no input from either bump sensor (input [L R] = [0 0]); both neurons are inactivated resulting in the output of [-1 -1] thus driving the wheels forward at the same rate.
• If the left bump sensor is triggered (input [L R] = [1 0]):
  • The drive left wheel neuron is activated as: (In1 x w11) + (In2 x w21) = (1 x 0.5) + (0 x 1) = 0.5
  • The drive right wheel neuron is activated as: (In1 x w12) + (In2 x w22) = (1 x 1) + (0 x 0.5) = 1
• If these values drive a PWM controller, the left wheel is driven in reverse at half the speed of the right wheel thus causing the robotic vehicle to reverse on a curved path.
• Once a period of time has passed, the robotic vehicle reverts to its default condition driving the vehicle forward in a straight line once again.

Common problems during implementation

The most common problem programmers face when building such as network is in the implementation of timers within the code, as opposed to in the implementation of the ANN.

If each neuron in the network is independent, it may switch on and off at its own time depending on when it is activated thus requiring an Independent Timer. Neurons may therefore become unsynchronised with other neurons which could result in an input pattern being implemented incorrectly by the ANN. One solution to this is for the ANN to maintain a Master Timer (aka. a System Timer), which should be updated on each cycle and neurons should defer back to at the start of each cycle. In Pseudocode, this may be implemented as follows:

System_Timer = 0;
Main_Program_Loop ()
    System_Timer = System_Timer + 1;
   
    % Neuron 1
    if N1 == ON
        N1_Timer = N1_Timer +1;
        % << Neuron 1 routine >>
        if N1_Timer == final_value
            reset N1_Timer
        end
    end
   
    % Neuron 2
    % etc.
Goto Main_Program_Loop

Summary

In summary:

• In this section, we introduced the Spiking Neuron and learned how its function may be implemented.
• We learned in what applications Spiking Neurons may be of benefit, in particular, in those applications that require timing control.
• We reviewed how a Spiking Neuron may be used as a PWM controller.
• Finally, we considered how a network that used PWM neurons could be used to implement simple obstacle avoidance for a robotic vehicle.